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SECTION  I 
INTRODUCTION 

Much  has  been  written  on  the  developing  engineering  sophistication  and  the 
student  centered  instructional  techniques  in  flight  simulator  design  and  utilization. 
Engineering  advances  combined  with  improved  training  strategies  place  the 
flight  simulator  in  contention  as  a major  flight  training  medium  in  today's 
military  environment.  There  is  an  increasing  awareness  that  simulators, 
efficiently  utilized,  can  be  employed  to  startling  advantage  in  military  flight 
training. 

The  Training  Analysis  and  Evaluation  Group  (TAEG)  is  examining  the  extent 
that  substitution  of  simulator  training  for  in-flight  training  is  feasible  in 
the  military  training  environment.  The  group  has  been  working  directly  with 
Patrol  Squadron  THIRTY  (VP-30)  to  maximize  the  use  of  existing  training  resources 
in  fleet  replacement  pilot  training  of  first-tour  aviators  in  the  P-3  aircraft. 

A recent  study^  evaluated  the  effectiveness  of  the  newly  installed  Device 
2F87F  Operational  Flight  Trainer  (OFT)  at  the  Fleet  Readiness  Squadron  (FRS), 

VP-30.  The  study  determined  the  training  and  cost  effectiveness  of  the  2F87F 
as  a replacement  for  the  earlier  generation  2F69D  OFT  when  used  in  combination 
with  the  P-3  aircraft. 

The  study  reported  here  is  a continuation  of  the  effort  to  integrate  the 
2F87F  into  the  ongoing  replacement  pilot  training  program.  Additional  data 
are  provided  to  aid  in  decisions  for  maximizing  the  role  of  the  2F87F  simulator 
in  the  production  of  P-3  pilots. 

PURPOSE 

The  present  study  continues  the  investigation  of  the  training  effectiveness 
of  Device  2F87F  by  examining  additional  factors  that  influence  device  utilization. 

The  specific  objectives  of  the  study  are  to  determine  the: 

performance  of  a group  trained  in  the  aircraft  without  previous 
simulator  training  to  permit  comparison  with  performance  of  matched 
groups  having  correlative  simulator  training, 

value  of  training  trials  for  providing  an  index  of  student  performance 
and  device  effectiveness, 

correlation  of  performance  in  Device  2F87F  with  performance  in  the 
P-3  aircraft, 

effect  of  undergraduate  pilot  training  (UPT)  performance  on  subsequent 
performance  in  FRS, 

performance  of  VP-30  trained  students  in  subsequent  operational 
assignments. 

^R.  F.  Browning,  L.  E.  Ryan,  P.  G.  Scott,  and  A.  F.  Smode.  Training  Effective- 
ness Evaluation  of  Device  2F87F,  P-3C  Operational  Flight  Trainer.  TAEG  Report 
No.  42.  Training  Analysis  and  Evaluation  Group,  Orlando,  FL.  ADA035771. 
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A subsequent  study  will  examine  the  influence  of  two  additional  major  ■ 

variables  on  the  training  effectiveness  of  Device  2F87F.  These  are  the  contribu-  ; 

tion  of  the  visual  system  to  performance  in  the  final  landing  phase  and  the 
effect  of  removing  simulation  motion  cues  on  the  transfer  of  training. 

PERSPECTIVE 

As  a prelude  to  the  discussions  which  follow,  several  issues  dealing  with 
real  world  contexts  should  be  noted.  To  begin  with,  the  TAEG  study  program  in 
the  P-3  FRS  community  centered  on  assessing  the  contributions  of  a recently 

installed  high  fidelity  flight  simulator  in  transitioning  pilots  for  assignment  i 

to  P-3  squadrons.  The  goal  of  the  program  was  to  efficiently  integrate  Device  j 

2F87F  into  the  ongoing  VP-30  pilot  training  program  and  to  reduce  P-3  aircraft 
in-flight  training  requirements. 

The  TAEG  studies  are  of  singular  interest  since  all  work  was  accomplished 
in  the  operational  environment.  Experimental  control  and  standardized  data 
collection  were  maintained  in  that  a TAEG  member  was  onsite  at  VP-30  during 
all  formal  studies.  Guidance  and  support  were  provided  to  instructor  pilots 
conducting  the  student  performance  evaluations.  Training  Analysis  and  Evaluation 
Group  members  observed  student  instruction  both  in  the  simulator  and  during 
aircraft  training  flights.  The  benefits  of  this  "in  situ"  approach  far 
outweigh  the  disadvantages  of  accommodating  confounding  influences  and  scheduling 
problems  in  the  environment  wherein  VP-30  conducted  business  as  usual.  The 
most  noteworthy  among  these  involved  data  gathering  constraints  and  range  of 
instructor  pilot  experience  levels.  The  latter  included  problems  arising  from 
instructor  pilot  rotation,  use  of  instructors  with  primary  duties  other  than 
flight  instruction,  and  the  biases  associated  with  utilizing  many  instructor 

pilots  in  evaluation  of  student  performance.  ^ 

Another  feature  of  importance  was  the  opportunity  to  systematically  i 

assess  the  performance  of  a group  of  students  trained  only  in  the  aircraft. 

This  initiative  is  seldom  exercised  in  studies  conducted  in  the  operational 
environment.  Training  such  a group  contributes  powerfully  to  the  study  design 
in  that  baseline  data  are  provided  for  assessing  simulator  contributions  to 
the  performance  of  groups  trained  in  both  the  simulator  and  the  aircraft.  A 
measure  is  provided  of  the  in-flight  training  required  in  the  absence  of  a 
simulator. 

Finally,  operational  implementation  of  a recommended  training  program  was 
achieved.  The  syllabus  of  instruction  used  for  the  experiments  with  modifications 
imposed  by  simulator  and  aircraft  availability  was  employed  for  three  consecutive 
FRS  classes.  This  phase  of  the  study  was  accomplished  by  squadron  personnel 
with  TAEG  in  a consulting  role.  Although  lacking  in  certain  experimental 
controls  and  rigor,  the  data  provide  additional  valuable  insights  for  assessing 
simulator  effectiveness  in  FRS  training. 

All  told,  the  onsite  measurement  of  simulator  contributions  to  P-3  Familiar- 
ization/Instrument (FAM/INST)  flight  training  afforded  a unique  opportunity 
for  highly  relevant  evaluations  within  a tolerable  range  of  experimental 
control . 
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ORGANIZATION  OF  THE  REPORT 

In  addition  to  this  introduction,  four  sections  are  presented 
II  presents  comparative  data  on  matched  groups  of  first-tour  pilots  '' 
principal  P-3  tasks  with  and  without  correlative  simulator  training 
formance  of  a group  of  students  trained  in  the  aircraft  without  prev. 
simulator  training  was  measured  to  establish  a baseline  for  compansoi 
performance  for  groups  trained  in  Device  2F37F  and  the  older  OFT,  Dev! 
2F69D. 

Section  III  describes  the  results  of  the  operational  implementaf 
the  experimental  simulator  and  flight  syllabus  for  three  entire  classt- 
the  concomitant  problems  of  scheduling  and  sharing  of  the  visual  system 
model.  Summary  data  related  to  training  trials  required  for  each  tas* 
examined  as  a source  of  additional  information  on  student  performance  a 
device  effectiveness. 

Section  IV  examines  additional  variables  that  presumably  influen. ^ 
outcomes  in  P-3  pilot  training.  The  relationships  between  DPT  flight 
UPT  flight  hours,  and  FRS  performance  are  analyzed.  The  feasibility  ♦ 
performance  in  the  aircraft  based  on  performance  in  the  flight  simulate 
explored.  Finally  the  results  of  a followup  questionnaire  on  pilots  . 
earlier  experiments  seeks  to  determine  if  any  differences  in  performa' 
between  experimental  and  control  students  exists  after  assignment  to  ar 
squadron. 

Section  V presents  conclusions  and  recommendations  developed  dunr. 
study. 
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SECTION  II 

COMPARISON  OF  AIRCRAFT  PERFORMANCE  FOR  SIMULATOR  AND 
NON-SIMULATOR  TRAINED  STUDENTS 

This  section  presents  data  on  pilots  trained  on  principal  flight  tasks 
in  the  P-3  aircraft  without  correlative  training  in  the  2F87F  flight  simulator. 
The  data  on  the  performance  of  this  group,  hereafter  referred  to  as  the  E-2 
group,  provide  a baseline  reference  measure  for  determining  the  value  or 
flight  hour  savings  of  alternative  mixes  of  simulator  and  aircraft  training. 
These  baseline  data  are  compared  with  the  data  from  groups  who  received  both 
simulator  and  aircraft  training.  The  data  were  collected  over  a period  of  8 
months. 

STUDY  DESIGN 

Three  matched  groups  were  identified  in  the  design:  a control  group 
trained  in  the  older  OFT,  Device  2F69D;  an  experimental  group  trained  in  the 
new  OFT,  Device  2F87F;  and  an  experimental  group  trained  in  the  P-3  aircraft 
without  prior  OFT  training.  Table  1 outlines  the  possible  comparisons. 

TABLE  1.  STUDY  DESIGN 


CONTROL  GROUP  (C) 
N=58 

EXPERIMENTAL  GROUP  (E-1) 

N=27 

FLY  ONLY  GROUP  (E-2) 
N=10 

4 CFT 

4 CFT 

4 CFT 

6 CPT 

6 CPT 

6 CPT 

3 2F69D 

6 2F87F 

— 

6 P-3  flights 

4 P-3  flights 

6 P-3  flights 
(minimum) 

TRAINING  TASKS.  Twenty-two  tasks  selected  by  TAEG  and  the  squadron  were  used 
as  the  basis  for  comparing  performance  of  the  three  groups  of  pilots.  This 
compares  to  20  check  tasks  for  the  earlier  group.  The  additional  tasks  were 
subsumed  under  other  tasks  in  the  earlier  study.  The  tasks,  identified  by 
circles  on  figure  1,  were  considered  most  appropriate  for  measurement  of 
pilot  skills  and  simulation  effectiveness  of  the  new  device. 

TRAINING  DEVICES  UTILIZED  IN  THE  STUDY.  Descriptions  of  the  two  part-task 
trainers  employed  with  all  groups  in  the  study;  the  older  operational  flight 
trainer.  Device  2F69D,  used  to  train  the  control  group;  and  the  newer  opera- 
tional flight  trainer.  Device  2F87F,  used  to  train  the  first  experimental 
group  are  provided  below. 
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Cockpit  Fami 1 iarization  Trainer  (CFT),  Device  2C23.  The  CFT  provides  a 
static  simulation  of  the  pilot,  copilot,  and  flight  engineer  positions.  It 
is  used  to  facilitate  the  learning  of  the  nomenclature,  location,  and  function 
of  the  various  controls,  instruments,  switches,  and  annunciator  lights.  The 
device  is  well  suited  to  the  learning  of  repetitive  tasks  such  as  normal  and 
emergency  procedures. 

Cockpit  Procedures  Trainer  (CPT),  Device  2C45.  The  CRT  was  developed  from  a 
modification  of  an  obsolete  P-3  OFT.  The  motion  simulation,  most  of  the 
flight  dynamics,  and  unneeded  systems  were  removed  or  disabled.  The  device 
in  its  present  configuration  provides  training  in  power  plant  management  and 
systems  procedures  for  both  normal  and  emergency  operations. 

Operational  Flight  Trainer,  Device  2F69D.  An  older  operational  flight  trainer 
configured  to  tne  ear'Iier  P-3A/B  models  was  used  in  the  training  of  the  con- 
trol group.  This  solid  state  analog  device,  which  was  the  principal  simulator 
used  before  delivery  of  the  2F87F,  came  into  the  inventory  late  in  1966  and 
provides  crew  or  individual  training  for  the  pilot,  copilot,  and  flight 
engineer.  The  2F69D  simulates  the  flight  dynamics,  systems,  navigation,  and 
communications  functions  of  the  P-3  aircraft  and  provides  limited  motion  (3 
degrees  of  freedom)  and  environmental  cues.  No  visual  simulation  is  provided. 
The  device,  with  its  analog  simulation,  requires  considerable  maintenance  to 
insure  high  fidelity  performance. 

Operational  Flight  Trainer,  Device  2F87F.  This  state  of  the  art  device 
simuTates  the  flight  stations  (pilot,  copilot,  and  flight  engineer)  of  the 
P-3C  Orion,  a four-engine  turboprop  aircraft  used  to  support  landbased  ASW 
and  other  long  range  surveillance  and  data  gathering  missions.  The  high 
fidelity  digital  device  is  equipped  with  a 6 degrees  of  freedom  motion  system 
and  a visual  capability  which  is  a narrow  angle  (50°  horizontal,  38°  vertical) 
television  rigid  model  system.  A broad  range  of  environmental  conditions 
varying  from  full  daylight  color  to  darkness  with  variable  visibility,  ceiling, 
and  wind  conditions  can  be  simulated.  The  model  board  simulates  an  area  of 
approximately  15  X 5 nautical  miles  on  a scale  of  2000  to  1 for  the  low 
altitude  maneuvers  associated  with  takeoff,  landing,  and  instrument  approaches. 
Low  altitude  on-top  conditions  are  simulated  electronically,  and  high  altitude 
simulation  is  provided  by  a high  altitude  model  board. 

SUBJECTS.  Ten  newly  designated  first-tour  naval  aviators  from  Class  76T03 
were  selected  as  subjects  for  the  "Fly  Only"  (E-2)  group.  This  group  was 
matched  on  the  basis  of  undergraduate  basic  and  advanced  flight  scores  with 
the  control  group  (C)  and  experimental  group  (E-1).  All  subjects  had  completed 
undergraduate  multiengine  training  in  the  S-2,  a small  twin  reciprocating 
engine  aircraft.  All  possessed  standard  instrument  cards. 

INSTRUCTORS.  The  most  experienced  VP-30  instructors  were  used  to  train  the 
E-2  group.  This  was  a safety  precaution  taken  to  offset  student  inexperience 
since  none  had  any  previous  training  in  the  2F87F  simulator.  Each  instructor 
was  briefed  by  TAEG  personnel  on  the  purpose  of  the  study,  the  proficiency- 
based  grading  system,  and  the  data  recording  requirements.  Flight  checks  for 
all  students  were  given  by  off-wing  instructors. 
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UNIVERSAL  GRADE  SHEET 


TRAINING  SESSION 


FLIGHT  TIME  TOTAL 


FIRST  PILOT  TIME 

SIMULATED  COPILOT  TIME 


UNSATISFACTORY  INCOMPLETE 


AA  A BA  U 


BRIEFING  TIME 


REMARKS  ON  BACK 


TRIALS 


(Ol)  PREFLIGHT 


@USE  CKLST  (CPT) 


(§)  ENGINE  STARTS 


04  START  MALF(CPT) 


@1NSTR  PROC 


07  ANTI  ICE  (CPT) 


@ TAKEOFF 


@ ABORT  4 ENG 


ABORT  3 ENG 


(|2)efar 


yL*. 

®ni 

i^r 


m 


ill 

lUB 


eSBH 


17  PROP  MALF(CPT) 


(CPT) 

13  EMERG  SHTWN 

19  ENG  RSTRT(CPT) 
CFT 

20  AIRCND/PRSR  OP 


21  HYD  SYS  OP(CPT) 


(CPT) 

22  FUEL  SYS  OP 


23  MAV  OMST  FAO: 


24  ELEC  SYS  OP{CPT 


25  FIRE  UNK  ORIG.  (CPT) 


26  SMOKE  REMOVAL  (CPT) 


27  RES  ELECT  PWR  (CPT) 


28  BAILOUT  DRILL  (CPT) 


29  EMERG  DESCENT  (SIM) 


30  DITCH  DRILL  (SIM) 


(31)  HOLDING 


@NON  PREC  APP 


(^PREC  APP 


34  CIRCLING  APP 


35  MISSED  APP 


(^LOG  PTRN  AIRWORK 


(^NORMAL  LANDINGS 


^APPROACH  FLAP  LOGS 


@WAVEOFF 


41  2 ENG  LDG 


4?)KNWLG  PROCEDURES 


44  COPILOT  RESP'S 


Figure  1.  Universal  Grade  Sheet 
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PROCEDURE 

GROUND  SCHOOL,  GET,  AND  CPT  TRAINING.  The  Fly  Only  (E-2)  group  received  the 
same  ground  school,  CFT,  and  CPT  training  as  the  control  (C)  and  the  2F87F 
trained  (E-1)  groups. 

FLY  ONLY  GROUP  IN-FLIGHT  TRAINING.  The  Fly  Only  (E-2)  group  received  in- 
flight training  in  the  same  tasks  as  the  control  (C)  and  the  2F87F  trained 
(E-1)  groups.  The  E-2  and  C groups  were  scheduled  for  six  P-3  flights  and 
the  E-1  group  for  four  flights.  However,  some  subjects  in  the  E-2  group 
required  up  to  eight  flights  to  satisfactorily  complete  the  flight  check. 

MEASUREMENT.  During  aircraft  flights  all  students  were  assigned  grades  based 
on  the  conventional  grading  system  used  in  Navy  pilot  training.  In  this 
system,  referred  to  as  the  "U,  BA,  A,  AA,"  (UBAA)  the  letter  U denotes  unsat- 
isfactory performance  and  is  equated  to  a numerical  grade  of  zero;  BA  denotes 
below  average  and  a grade  of  2.5;  A denotes  average  and  a grade  of  3.0;  and 
AA  denotes  above  average  and  a grade  of  3.5.  The  numerical  scores  of  all 
students  were  compiled  and  averages  obtained  for  individuals  and  for  the 
group. 

For  the  purposes  of  the  study  a second  measurement  system  based  on 
attainment  of  proficiency  in  each  task  trained  was  used.  Proficiency  (P)  was 
defined  as  performance  estimated  to  be  equivalent  to  that  required  to  demon- 
strate competence  on  the  conventional  flight  check.  The  proficiency  measurement 
system  was  used  in  both  the  simulator  and  the  aircraft.  Instructors  assigned 
a "P"  to  each  task  when  it  was  performed  to  proficiency  in  the  simulator  and 
again  when  it  was  performed  to  proficiency  in  the  aircraft.  Proficiency  was 
assumed  for  any  task  graded  "A"  or  "AA"  on  the  flight  check. 

RESULTS 

The  data  are  presented  under  two  main  topics:  (1)  Actual  Flight  Training 
Hours  and  (2)  Proficiency-Based  Flight  Training  Hours.  The  actual  flight 
training  hours  are  the  average  number  of  flight  hours  received  by  the  C,  E-1, 
and  E-2  group  students.  The  proficiency-based  flight  training  data  represent 
the  number  of  flights  required  to  attain  proficiency  on  the  designated  check 
tasks. 

ACTUAL  FLIGHT  TRAINING  HOURS.  Table  2 presents  summary  data  on  the  three 
groups  identified  in  the  study.  Undergraduate  pilot  training  flight  averages, 
average  VP-30  flight  hours,  and  VP-30  flight  averages  are  shown. 

The  data  of  most  interest  concern  the  flight  hour  comparisons  among  the 
three  groups.  The  first  pilot^  flight  hours  for  the  C and  E-2  groups  are 
identical  (15.1  hours)  which  indicates  that  Device  2F69D,  as  utilized  during 
this  study,  was  not  contributing  to  a reduction  in  flight  hours.  The  8.6  hours 


FAM/INST  training  at  VP-30  is  directed  toward  first  pilot  training  (left 
seat)  and  only  tasks  performed  in  this  position  are  graded. 
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TABLE  2.  AVERAGE  FLIGHT  HOURS  AND  FLIGHT  GRADES  OF  CONTROL 
AND  EXPERIMENTAL  GROUPS 


Device  2F69D  & 
Flight  Training 

Device  2F87F  & 

FI  ight  Training 

Flight  Training 
Only 

C 

E-1 

E-2 

Number  of  Students 

58 

27 

10 

Flight  Average  (UPT) 

55.8 

54.2 

55.0 

VP-30  Flight  Hours 

Per  Student 

' 15.1 

8.6 

15.1 

VP-30  Check  Flight 

Average  Grade 

3.02 

3.03 

3.01 

received  by  the  E-1  group  represent  a 43  percent  savings  over  both  the  C and 
E-2  groups.  The  flight  hour  savings  are  attributed  to  the  effective  utiliza- 
tion of  Device  2F87F. 


Savings  in  flight  time  is  a good  measure  of  the  effectiveness  of  a 
training  device.  Another  way  of  depicting  simulator  effectiveness  is  via  the 
computation  of  the  transfer  effectiveness  ratio  (TER). 3 

The  TER  between  the  E-1  and  E-2  groups  is  computed  below  and  is  provided 
as  another  way  of  displaying  the  findings  of  the  study. 

TER  = Flight  Hours*  (E-2)  - Flight  Hours*  (E-1) 

Simulator  Hours*  (E-1) 

TER  = 15.1  - 8.6  = .54 

T2 

^First  Pilot  Hours  Only 

The  TER  value  indicates  the  hours  of  flight  time  saved  for  every  hour  of 
training  in  the  simulator.  The  reader  is  cautioned  not  to  interpret  the  .54 
TER  as  a constant.  The  TER  is  not  necessarily  linear  with  increased  training 
and  it  varies  as  a function  of  the  tasks  trained  and  the  extent  of  previous 
practice. 


^ S.  N.  Roscoe.  "Incremental  Transfer  Effectiveness."  Human  Factors.  1_3«  6. 
December  1971.  pp.  561-567. 
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In  addition  to  a comparison  of  flight  hours  required  to  train  the  three 
groups,  table  2 presents  the  average  check  flight  scores,  the  number  of  students 
in  each  group,  and  their  average  DPT  flight  grades.  Although  there  are  slight 
differences  in  the  VP-30  check  flight  grades,  these  differences  are  not 
significant. 

PROFICIENCY-BASED  TRAINING  HOURS,  The  following  five  tables  provide  data  based 
on  the  proficiency  (P)  grading  system.  Table  3 presents  the  cumulative  pro- 
portion of  check  tasks  on  which  the  E-1  and  E-2  group  trainees  were  judged 
proficient  in  the  airplane. 

TABLE  3.  CUMULATIVE  PROPORTION  OF  CHECK  TASKS  ON  WHICH  EXPERIMENTAL  GROUPS 
WERE  JUDGED  PROFICIENT  IN  THE  AIRCRAFT 


FLY  1 

FL  2 

FL  3 

FL  4 

FL  5 FL  6 FL  7 FL  8 

Tasks  trained  to  proficiency 
in  Device  2F87F  (E-1  Group) 

.76 

.87 

.94 

.99 

.99* 

Tasks  practiced  in  Device 
2F87F  but  not  trained  to 

proficiency  (E-1  Group) 

.46 

.60 

.75 

.96 

1.00* 

Fly  Only  Group  (E-2  Group) 

.09 

.34 

.52 

.57 

.83  .91  .94**  .95** 

* 1 student  required  5 flights  to  achieve  proficiency. 

**  3 students  required  7 flights  and  1 student  required  8 flights  to  achieve 
proficiency. 


The  experimental  design  for  the  E-1  group  called  for  all  20  check  tasks  to  be 
performed  on  FLY  1.  For  various  reasons  (e.g. , maintenance  problems,  weather, 
instructor  oversight),  the  actual  number  of  tasks  checked  on  FLY  1 varied 
from  9 to  19.  Similarly,  for  the  E-2  group,  the  number  checked  on  FLY  1 
ranged  from  7 to  15  from  a total  of  22  check  tasks.  This  lower  range  of 
tasks  presented  was  expected  for  the  E-2  group  since  this  group  had  to  achieve 
certain  task  skills  without  previous  simulator  exposure  prior  to  attempting 
more  complex  tasks  in  the  aircraft.  The  average  number  of  tasks  presented  to 
the  E-2  group  on  FLY  1 was  11.7,  By  FLY  4 the  average  number  presented  was 
20.2,  However,  the  trainees  were  judged  proficient  in  only  57  percent  of  the 
tasks.  The  simulator  trained  group  (E-1)  did  much  better  by  FLY  4;  trainees 
were  judged  proficient  in  96  percent  of  the  tasks  if  they  had  received  some 
practice  in  the  simulator  and  proficient  in  99  percent  of  the  tasks  if  training 
in  the  simulator  was  to  proficiency.  This  offers  additional  evidence  that 
the  training  of  check  tasks  to  proficiency  in  the  2F87F  prior  to  in-flight 
training  reduces  the  time  for  these  tasks  to  be  judged  proficient  in  the 
aircraft.  The  data  also  show  that  in  terms  of  proficiency  attainment  the  E-2 
group  was  not  as  well  prepared  for  the  flight  check  nor  did  they  perform  as 
well  as  the  E-1  group  on  the  flight  check. 


16 


TAEG  Report  No.  54 


Table  4 presents  a comparison  of  the  attainment  of  proficiency  on  check 
tasks  for  the  E-1  and  E-2  groups.  Data  in  this  table  are  based  on  the  assump 
tion  that  a check  task  presented  for  the  first  time  on  FLY  1,  2,  3,  or  4 and 
judged  proficient  on  that  flight,  required  only  that  one  flight  to  be  judged 
proficient  and  was  scored  as  proficient  on  FLY  1. 


TABLE  4.  CHECK  TASK  PROFICIENCY  ATTAINMENT 


EXPERIMENTAL 

EXPERIMENTAL 

GROUP  E-1 

GROUP  E-2 

N=27 

N=10 

AVERAGE  FLIGHTS 

AVERAGE  FLIGHTS 

CHECK  TASKS 

TO  PROFICIENCY 

SO 

TO  PROFICIENCY 

SD 

Preflight 

1.4 

.84 

2.5 

.85 

Use  of  Checklists 

1.1 

.27 

1.9 

.88 

Engine  Starts 

1.0 

.00 

1.6 

.84 

Taxi 

1.1 

.42 

2.6 

.97 

Instrument  Procedures 

1.6 

.89 

4.3 

1.33 

Brake  Fire 

1.1 

.24 

1.5 

.53 

Takeoff 

1.1 

.32 

3.0 

1.25 

Abort  Four  Engines 

1.1 

.29 

2.0 

.94 

Abort  Three  Engines 

1.4 

.75 

1.8 

.63 

Engine  Failure  After  Refusal  1.4 

.69 

2.3 

.82 

Departure 

1.1 

.20 

2.8 

1.03 

Basic  Airwork 

1.6 

i.bi 

3.7 

1.77 

Holding 

* 

1.6 

.73 

Non-precision  Approach 

1.3 

.60 

2.7 

1.42 

Precision  Approach 

1.4 

.69 

2.9 

1.29 

Landing  Pattern  Airwork 

1.7 

.92 

3.3 

1.70 

Normal  Landings 

1.7 

.94 

4.6 

1.78 

Approach  Flap  Landings 

# 

2.0 

1.05 

Waveoff 

1.2 

.40 

3.1 

1.52 

Three  Engine  Landings 

1.7 

.91 

2.4 

1.45 

No  Flap  Landings 

1.6 

.74 

1.8 

.79 

Knowledge  of  Procedures 

1.4 

.79 

4.6 

2.01 

* Included  in  other  phases  of  instruments. 

# Included  under  normal  landings. 


The  column  labeled.  Average  Flights  to  Proficiency,  represents  the 
number  of  flights  the  students  flew  in  the  P-3  before  being  judged  proficient 
for  that  task. 

For  every  task,  proficiency  in  the  aircraft  was  attained  in  fewer  flights 
for  the  E-1  group  than  for  the  E-2  group.  A task-by-task  comparison  indicates 
the  benefits  of  Device  2F87F  training  to  be  the  greatest  for  (1)  Knowledge  of 
Procedures,  (2)  Normal  Landings,  (3)  Instrument  Procedures,  and  (4)  Basic 
Airwork.  The  beneficial  effects  of  2F87F  training  for  Knowledge  of  Procedures 
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and  Normal  Landings  were  also  reported  in  a previous  study, ^ The  findings  of 
this  study  indicate  that  Device  2F37F  is  as  effective  for  training  the  more 
difficult  tasks  as  it  is  for  training  knowledge  and  procedural  tasks.  In  21 
of  the  22  tasks  the  standard  deviation  (SD)  for  the  group  trained  only  in  the  air- 
craft (E-2)  was  greater  than  for  the  group  trained  in  the  simulator  and  the 
aircraft  (E-l).  The  data  indicate  that  variability  in  task  performance  is 
less  for  simulator  trained  students  than  for  those  not  trained  in  the  simulator. 

Table  5 shows  the  number  of  check  tasks  presented  and  the  number  of 
check  tasks  on  which  the  E-2  group  trainees  were  judged  proficient.  Table  6 
presents  the  same  information  for  the  E-l  group.  The  table  is  reproduced 
from  the  previous  study^  and  included  here  solely  for  comparison  purposes. 

Four  students  from  group  E-2  (7,  8,  9,  and  10)  required  five  flights  to 
become  proficient  on  all  check  tasks  (average  of  12.7  flight  hours  per 
student).  Three  students  (3,  4,  and  5)  were  proficient  after  six  flights 
(13.8  hours).  Two  students  (1  and  2)  were  proficient  after  seven  flights 
(15.95  hours)  and  one  student  (6)  required  eight  flights  (17.7  hours)  to 
become  proficient  in  all  check  tasks.  The  average  flight  time  for  all  students 
to  attain  proficiency  was  14.2  hours.  This  compares  to  an  average  of  6.2 
flight  hours  required  by  the  E-l  group  to  become  proficient.  Based  on  the 
flight  hours  to  proficiency,  the  TER  is  computed  as  follows: 


.rrn  _ Flight  Hours  to  Proficiency  (E-2)  - Flight  Hours  to  Proficiency  (E-l) 

Simulator  Hours  (E-l) 

^rn  _ 14.2  - 6.2  = .67 
ter ^ 

The  TER  of  .67  is  greater  than  that  obtained  by  comparing  actual  flight 
hours  (.54  TER).  It  is  viewed  as  a better  estimate  of  the  training  effective- 
ness of  the  2F87F  since  the  use  of  proficiency  hours  represents  a comparison 
of  criterion  referenced  performance  levels,  whereas  the  use  of  actual  hours 
represents  a comparison  of  end-of-program  performance  levels. 

The  reliability  of  the  proficiency-based  grading  system  is  attested  to 
by  the  finding  that  in  only  22  out  of  1020  gradings  were  students  subsequently 
given  a grade  below  average  on  a task  that  had  previously  been  judged  proficient. 
This  compares  favorably  with  the  findings  from  the  previous  TAEG  studyo  where 
only  50  out  of  1200  gradings  were  students  subsequently  graded  below  average 
on  a task  previously  judged  proficient. 


4 

Browning, 

^ Ibid. 

^ Browning, 
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TABLE  6.  NUMBER  OF  CHECK  TASKS  PRESENTED  AND  NUMBER  OF  CHECK  TASKS 
ON  WHICH  E-1  TRAINEES  WERE  CERTIFIED  PROFICIENTI 
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NOTE:  Each  "FLY"  represents  2.11  flight  hours.  Only  two  students  required  five  flights. 
1 Taken  from  TAEG  Report  No.  42 
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LANDING  PERFORMANCE 

Great  concern  is  devoted  to  the  landing  task  in  P-3  flight  training,  and 
nx)re  time  is  spent  in  training  this  skill  than  any  other  task  in  the  syllabus. 
Concomitant  with  the  concern  is  the  belief  by  squadron  instructors  and  pilots 
that  landing  practice  in  Device  2F87F  does  not  transfer  to  the  aircraft 
because  the  device  does  not  realistically  simulate  the  aircraft  performance 
during  the  final  landing  phase.  The  most  prominent  complaint  is  the  lack  of 
peripheral  cues  (purportedly  required  for  landing)  attributed  to  the  narrow 
field  of  view  of  the  visual  system.  The  design  of  this  study  provided  an 
opportunity  to  test  this  contention. ^ A comparison  was  made  of  the  number  of 
landings  required  to  achieve  proficiency  and  the  number  of  landings  actually 
received  by  both  the  simulator  and  the  aircraft  trained  groups.  Table  7 
presents  the  average  numbc'^  of  landings  required  by  both  groups  to  attain 
proficiency  and  the  number  actually  received. 

TABLE  7.  AVERAGE  NUMBER  OF  LANDINGS 


Aircraft  Landings 

Device  2F37F 

Landings 

Actual 

To  Proficiency 

E-1 

28* 

36 

17 

E-2 

— 

60 

50 

*Estimated  from  computer  printouts 


The  E-2  group  required  24  more  landings  per  student  to  complete  the  FAM/ 
INST  phase  of  fleet  readiness  training  and  33  more  landings  per  student  to 
attain  proficiency  in  landings.  It  is  interesting  to  note  that  the  simulator 
trained  group  (E-1)  required  a combination  of  45  landings  (28  simulator  and 
17  aircraft)  to  achieve  proficiency  whereas  the  Fly  Only  group  (E-2)  required 
50  aircraft  landings  to  achieve  proficiency. 

SUMMARY  OF  FINDINGS 

The  findings  discussed  in  this  section  are  summarized  below: 

1.  The  older  (Analog)  Device  2F69D  as  utilized  during  the  period  of  this 
study  and  immediately  prior  to  acceptance  of  Device  2F87F  did  not  provide 
significant  transfer  of  training  to  the  aircraft  for  dynamic  flight  tasks. 

^ A subsequent  TAEG  study  will  address  transfer  of  training  in  the  final  phase 
of  landing.  Students  will  be  trained  in  the  landing  pattern  task  in  the 
simulator.  However,  the  task  will  be  terminated  at  the  "Select  Land  Flaps" 
position  by  either  "freezing"  the  trainer  or  initiating  a waveoff. 
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2.  Students  trained  in  the  aircraft  without  prior  OFT  training  (the 
E-2  group)  required  an  average  of  6.5  more  flight  hours  in  the  P-3  aircraft 
to  complete  the  FAM/INST  phase  of  training  than  did  the  E-1  group.  With  the 
additional  flight  hours  their  flight  grades  were  lower  than  the  grades  of 
pilots  trained  in  Device  2F87F  and  the  P-3  aircraft. 

3.  Based  on  actual  flight  training  hours  received  the  TER  for  Device 
2F87F  is  .54  (for  every  hour  in  the  simulator,  .54  hours  of  flight  time  are 
saved).  When  a proficiency  grading  system  is  used  the  TER  is  increased  to 
.67.  This  suggests  that  if  training  on  each  task  is  terminated  upon  reaching 
proficiency,  1 hour  of  simulator  time  would  substitute  for  .67  hours  of 
flight  time. 

4.  The  Fly  Only  group  received  24  more  aircraft  landings  than  the 
group  receiving  simulator  and  aircraft  training.  They  required  33  more 
landings  to  achieve  proficiency  (table  7).  It  should  be  noted  that  the 
simulator  group  required  fewer  total  simulator  and  aircraft  landings  to 
attain  proficiency  than  did  the  aircraft  only  trained  group.  This  suggests, 
that  the  task  learned  in  the  simulator  transfers  significantly  to  subsequent 
aircraft  landing  performances. 
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SECTION  III 

TRAINING  TRIALS  AS  AN  INDEX  OF  STUDENT  PERFORMANCE 
AND  DEVICE  EFFECTIVENESS 

During  the  conduct  of  the  P-3  study  program,  it  became  increasingly 
clear  that  training  trials  data  (i.e,,  data  on  individual  training  trials) 
would  provide  important  information  about  student  performance  and  could 
provide  valuable  inputs  towards  determining  the  training  effectiveness  of 
Device  2F87F. 

The  opportunity  to  gather  training  trials  data  on  tasks  for  both  first- 
and  second- tour  students  emerged  with  the  acceptance  of  a second  Device  2F87F 
at  VP-30.  The  data  were  collected  by  VP-30  instructor  pilots  after  a brief 
indoctrination  by  TAEG  personnel.  The  Training  Analysis  and  Evaluation  Group 
did  not  monitor  the  collection  effort  but  did  conduct  all  data  analyses. 

This  phase  of  the  study  lacked  in  certain  experimental  controls  since 
the  data  were  gathered  solely  by  squadron  personnel  during  day-to-day  operations. 
However,  the  data  are  of  sufficient  substance  and  value  to  provide  insights 
concerning  training  strategies,  grading  criteria,  program  control,  and  the 
value  of  Device  2F87F  for  training  individual  tasks. 

DATA  COLLECTION 

Data  for  this  phase  of  the  study  were  collected  from  three  consecutive 
classes  of  students  (classes  7703,  7704,  and  7705),  hereafter  referred  to  as 
the  "0"  group.  The  grade  sheets  modified  to  collect  task  trials  (figure  2) 
were  completed  on  each  student  and  forwarded  to  TAEG  for  analysis. 

MEASUREMENT.  In  addition  to  the  "UBAA"  grading  system,  the  previously 
defined  proficiency  grading  system  was  used.  Proficiency  (P)  was  defined  as 
performance  estimated  to  be  equivalent  to  that  required  to  demonstrate  compe- 
tence on  the  conventional  flight  check. 

For  tasks  2,  3,  4,  6,  13,  15,  35,  36,  and  43  (see  figure  2),  P was 
assigned  by  the  instructor  in  accordance  with  the  proficiency  definition 
stated  above.  For  tasks  8,  9,  10,  11,  12,  31,  32,  32A,  32B,  32C,  33,  33A, 

37,  38,  39,  40,  and  42,  trial  performance  was  recorded  by  the  instructor  as 
either  a 1 or  a P,  and  TAEG  determined  the  point  at  which  P was  attained  for 
each  task.  The  procedure  for  making  this  determination  was  as  follows:  the 
trial  performance  was  recorded  by  the  instructor  as  "1"  (meaning  one  trial) 
or  "P"  (meaning  one  trial  that  was  proficient).  For  example,  10  normal 
landings  on  any  flight  might  have  been  graded  llPllPPlll;  of  the  10  trials,  7 
were  not  proficient  and  3 were  proficient.  The  rule  used  by  TAEG  for  deter- 
mining the  point  when  P was  attained  is  as  follows:  (1)  over  50  percent  of 
the  trials  (for  a given  task)  on  any  flight  had  to  be  P and  (2)  at  least  50 
percent  of  the  trials  were  P on  all  subsequent  flights.  An  exception  to  (1) 
and  (2)  could  occur  on  the  check  flight.  If  on  the  aircraft  check  flight  a UBAA 
grade  of  A or  AA  was  assigned  by  the  instructor,  then  P was  assigned  by  TAEG 
no  matter  how  the  individual  trials  were  graded.  Attainment  of  proficiency 
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Figure  2.  Universal  Grade  Sheet  (Revised) 
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for  a task  was  determined  by  TAEG  to  reduce  the  inconsistencies  occurring 
when  P is  determined  by  a number  of  instructors  and  to  standardize  this 
difficult  measurement. 

SEQUENCE  OF  TRAINING.  Instructors  were  briefed  to  train  all  tasks  to  profi- 
ciency in  the  simulator  before  proceeding  to  the  aircraft  whenever  possible. 
This  goal  was  not  uniformly  reached,  due,  in  part,  to  a revised  sequence  of 
training  (see  table  8)  which  was  dictated  by  aircraft  availability. 


TABLE  8.  SEQUENCE  OF  SIMULATOR  AND  FLIGHT  TRAINING  FOR  THE  0 GROUP 
(CLASSES  7703,  7704,  AND  7705) 


NOTE:  Due  to  aircraft  availability; 

*FLY  1 followed  SIM  5 for  some  students, 

**FLY  2 followed  SIM  6 for  some  students,  and 

***FLY  3 preceded  SIM  6 for  some  students. 

The  acceptance  of  the  second  flight  simulator  required  that  two  cabs 

share  the  visual  system  low  altitude  model  board.  Although  sharing  of  the 
model  board  did  not  reduce  training  time  for  tasks  requiring  visual  simulation 
by  50  percent  (as  one  cab  can  use  the  electronically  generated  horizon  scene 
for  high  altitude  work),  it  may  have  had  some  effect  on  training.  Additionally, 
conflicts  in  sharing  the  model  board  resulted  in  some  students  not  receiving 
equal  time  on  the  board.  The  effect  of  this  variable  was  not  isolated  during 
the  study. 

ANALYSIS  AND  RESULTS 

The  data  are  presented  under  three  topics;  (1)  Actual  Flight  Training 
Hours  Received,  (2)  Proficiency-Based  Training,  and  (3)  Simulator  and  Aircraft 
Task  Trial  Data. 

FLIGHT  TRAINING  HOURS 

Table  9 provides  a summary  of  the  performance  of  the  0 Group  and  compares 
this  performance  to  that  of  the  earlier  experimental  group  (E-1). 
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TABLE  9.  COMPARISON  OF  PERFORMANCE  FOR  0 AND  E-1  GROUPS 


0 Groups 

Combined  0 Group 

E-1  Group 

7703 

7704 

7705 

Number  of  Students 

14 

17 

8 

39 

27 

UPT  FI ight  Average 

56.9 

49.9 

49.3 

52.3 

54.2 

VP-30  Flight  Hours 
per  Student 

9.8 

9.7 

9.2 

9.6 

8.6 

VP-30  Average  Check 

FI ight  Grade 

3.02 

3.00 

2.96 

3.00 

3.03 

The  average  flight  hours  for  the  0 group  to  complete  the  FAM/INST  phase 
are  9.6,  an  increase  of  1 hour  over  the  E-1  group.  This  difference  could 
be  reasonably  due  to  a number  of  factors,  specifically; 

change  in  student  input  quality  (increased  variance  in  UPT  scores), 

O 

degradation  of  Device  2F87F  simulation  quality, 
more  difficult  criterion  for  attainment  of  proficiency, 
instructor  inexperience, 

. change  in  training  sequence;  i.e.,  integrated  vs.  block  training, 
and 

. failure  to  train  to  proficiency  in  Device  2F87F. 

Unfortunately,  the  specific  impact  of  each  of  these  variables  is  not  known. 
However,  subsequent  discussion  will  consider  these  variables,  as  appropriate. 

As  shown  in  table  9,  the  UPT  flight  average  of  the  0 group  is 
not  significantly  different  from  the  E-1  group,  but  the  UPT  flight  averages 
for  two  of  the  classes  included  in  the  0 group  are  significantly  lower.  The 
relationship  of  undergraduate  flight  scores  to  undergraduate  flight  hours  and 
to  later  performance  at  VP-30  is  discussed  in  section  IV  of  this  report.  The 
VP-30  flight  check  average  for  the  0 group,  3.00,  is  not  significantly  dif- 
ferent from  that  of  the  control  and  previous  experimental  groups  ■ 1.27). 


O 

Simulators  were  beset  by  a number  of  maintenance  problems  during  the  period 
of  0 group  training.  VP-30  has  since  established  a policy  of  not  accepting 
the  simulator  for  training  if  essential  simulation  is  unusable. 


26 


TAEG  Report  No.  54 


PROFICIENCY-BASED  TRAINING 

Figure  3 presents  data  on  proficiency  attainment  in  the  P-3  aircraft  as 
a function  of  proficiency  attainment  in  the  simulator.  0 group  students 
trained  to  proficiency  in  the  Device  2F87F  (0  "P")  performed  better  in  the 
aircraft  than  0 group  students  not  trained  to  proficiency  in  the  simulator 
(0  "NO  P").  This  relationship  also  holds  true  for  the  E-1  "P"  and  "NO  P" 
students.  Flowever,  the  probability  of  demonstrating  proficiency  in  the 
aircraft  on  the  first  flight  that  a task  was  introduced  was  considerably 
lower  for  the  0 group  compared  to  the  E-1  group  (.44  vs.  .76  and  .21  vs. 

.46).  Previously  mentioned  factors  such  as  the  effect  of  an  integrated 
simulator/flight  syllabus,  more  stringent  proficiency  standards,  changes  in 
student  quality,  difference  in  instructor  experience  levels,  and  failure  to 
award  proficiency  grades  in  accordance  with  established  criteria  could  be 
responsible  for  the  apparent  decrease  in  performance.  Whatever  the  cause, 
the  data  support  earlier  conclusions  that  students  should  be  trained  to 
proficiency  in  each  task  in  the  simulator  before  receiving  in-flight  training. 
Additional  support  for  this  conclusion  is  provided  by  a comparison  of  the 
Fly  Only  group  (E-2)  performance  with  performance  of  groups  E-1  and  0. 

Proficiency  attainment  of  the  Fly  Only  group  is  lower  than  that  of  the  E-1 
and  0 groups  across  all  flights.  This  evidence  suggests  that  some  training 
in  the  simulator  (even  if  not  to  proficiency)  is  better  than  no  simulator 
training. 

Unfortunately,  due  to  differences  in  student  learning  rates,  a proficiency- 
based  training  strategy  in  an  integrated  simulator/flight  syllabus  creates 
problems  in  scheduling.  These  problems,  however,  are  resolvable.  One  solution 
is  to  pair  incoming  students  based  on  their  demonstrated  performance  in 
undergraduate  pilot  training.  By  pairing  students  in  accordance  with  demonstrated 
abilities,  both  the  simulator  and  flight  syllabi  could  be  modified  to  coincide 
with  student  requirements.  The  more  able  student  would  not  be  held  to  the 
pace  of  the  less  able  nor  would  the  less  able  be  pushed  to  complete  the 
simulator  and  flight  syllabus  in  the  number  of  periods  required  for  the 
average  or  above  average  student. 

PROFICIENCY  ATTAINMENT  IN  THE  AIRCRAFT  ON  INDIVIDUAL  TASKS.  Table  10  shows 
the  cumulative  proportion  of  individual  check  tasks  on  which  students  were 
judged  proficient  in  the  aircraft  differentiated  by  whether  or  not  proficiency 
had  first  been  attained  in  the  flight  simulator.  As  expected,  the  benefits 
of  training  to  proficiency  in  the  simulator  are  most  noticeable  on  Fly  1 and 
Fly  2.  The  differences  in  proficiency  attainment  are  diminished  as  both 
groups  approach  asymptotic  performance. 

Table  10  highlights  apparent  deficiencies  in  the  training  received  by 
the  0 group.  As  shown,  the  performance  (probability  of  proficiency)  on  Fly  1 
was  below  .50  for  13  of  the  check  tasks  that  had  been  trained  to  proficiency 
in  the  simulator.  Considering  the  CFT,  CPT,  and  OFT  training  that  preceded 
Fly  1,  proficiency  attainment  could  reasonably  be  expected  to  be  higher  than 
.50.  For  example,  such  tasks  as  preflight,  use  of  checklists,  and  engine 
starts  should  have  been  trained  to  proficiency  before  leaving  the  CPT.  These 
skills  should  have  been  refreshed,  reinforced,  and  checked  for  proficient 
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Note:  Data  in  this  figure  are  based  on  the  assumption  that  a check  task 
presented  for  the  first  time  on  Fly  1,  2,  3,  or  4 and  judged 
proficient  on  that  flight  required  only  one  flight  to  be  judged 
proficient. 

"P"  = Trained  to  proficiency  in  2F87F. 

"No  P"  = Not  trained  to  proficiency  in  2F87F. 


Figure  3.  Cumulative  Attainment  of  Proficiency  in  the  Aircraft  as 
a Function  of  Proficiency  Attainment  in  the  Simulator 
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TABLE  10.  CUMULATIVE  PROPORTION  OF  INDIVIDUAL  CHECK  TASKS  ON  WHICH  STUDENTS 
FROM  THE  COMBINED  0 GROUP  WERE  JUDGED  PROFICIENT  IN  THE  AIRCRAFT 


CUMULATIVE 

PROPORTION  OF  CHECK  TASKS 

JUDGED 

PROFICIENT 

FLY 

1 

FLY 

2 

FLY 

3 

FLY 

4 

Task 

Task 

P in 

Not  P 

P in 

Not  P 

P in 

Not  P 

P in 

Not  P 

No. 

SIM 

in  SIM 

SIM 

in  SIM 

SIM 

in  SIM 

SIM 

in  SIM 

2. 

Preflight 

.43 

.11 

.77 

.29 

.85 

.75 

. 

1.00 

3. 

Use  of  Checklists 

.58 

.36 

.84 

.58 

1.00 

.82 

- 

1.00 

4. 

Engine  Starts 

.57 

.50 

.86 

.70 

.93 

.95 

1.00 

.95 

6. 

Taxi 

.33 

.14 

.75 

.27 

.75 

.67 

.75 

1.00 

8. 

Brake  Fire 

1.00 

.55 

- 

1.00 

- 

. 

- 

- 

9. 

Takeoff 

.24 

.25 

.73 

.53 

.86 

.86 

1.00 

1.00 

10. 

Abort  Four  Engines 

.53 

.17 

.91 

.55 

- 

.86 

- 

- 

11. 

Abort  Three  Engines 

.50 

.21 

.88 

.48 

- 

.89 

- 

- 

12. 

Engine  Failure  After 
Refusal 

.31 

.07 

.62 

.54 

.92 

.91 

13. 

Departure 

.41 

.37 

.64 

.53 

.81 

.86 

.95 

1.00 

15. 

Basic  Airwork 

.43 

.18 

.50 

.32 

.63 

.59 

1.00 

.96 

31. 

Holding 

.54 

.38 

.83 

.62 

1.00 

.86 

- 

- 

32. 

a.  TACAN/VOR 

.41 

0 

.87 

.14 

.93 

.65 

- 

.88 

b.  NDB 

- 

.18 

- 

.50 

- 

- 

- 

- 

c.  LOC 

1.00 

.28 

- 

.64 

- 

.78 

- 

.88 

33. 

a.  GCA 

.28 

.08 

.59 

.17 

.94 

.75 

- 

.87 

b.  ILS 

1.00 

.63 

- 

- 

- 

- 

- 

- 

35. 

Instrument  Procedures 

.67 

.11 

.75 

.29 

.92 

.68 

1.00 

.97 

36. 

Landing  Pattern  Airwork 

.25 

.11 

.75 

.52 

.75 

.77 

1.00 

.97 

37. 

Normal  Landings 

.31 

.07 

.38 

.12 

.92 

.42 

1.00 

.79 

38. 

Approach  Flap  Landings 

.81 

.28 

.90 

.56 

1.00 

.75 

- 

.91 

39. 

Waveoff 

.56 

.43 

.75 

.59 

1.00 

.80 

- 

.88 

40. 

Three  Engine  Landings 

.20 

.14 

.60 

.36 

.90 

.91 

• 

1.00 

42. 

No  Flap  Landings 

.27 

.22 

.91 

.76 

- 

1.00 

- 

- 

43. 

Knowledge  of 

Procedures 

.29 

.16 

.60 

.31 

.86 

.54 

1.00 

.85 

- = Not  presented. 
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performance  in  the  OFT.  Instrument  skills  such  as  holding,  instrument  proce- 
dures, precision  and  non-precision  approaches  plus  knowledge  of  procedures, 
particularly  those  that  had  been  trained  to  proficiency  in  Device  2F87F, 
should  only  require  validation  in  the  aircraft.  Based  on  performance  on  Fly 
1,  much  of  the  CPT  and  OFT  training  was  not  effective,  did  not  transfer,  or 
was  forgotten  prior  to  aircraft  training.  Past  research  has  adequately 
demonstrated  the  effectiveness  of  synthetic  devices  for  training  these  tasks. 

It  is  therefore  concluded  that  the  training  provided  for  this  group  was  not 
as  effective  as  it  might  have  been. 

The  tasks  listed  in  table  10  that  benefit  most  on  Fly  1 and  Fly  2 from 
proficiency  training  in  the  simulator  are  (1)  Instrument  Procedures,  (2) 

Aborts,  (3)  Precision  and  Non-precision  Approaches,  and  (4)  Landings.  For 
the  Normal  Landing  Task,  all  students  trained  to  proficiency  in  the  simulator 
reached  proficiency  in  the  aircraft  by  Fly  4.  Although  other  tasks  were  also 
judged  proficient  for  all  students  by  Fly  4,  the  difference  between  the 
proficiency  attainment  for  the  "P  in  the  simulator"  students  and  the  "Not  P 
in  the  simulator"  students  was  most  pronounced  for  the  normal  landing  task. 

SIMULATOR  AND  AIRCRAFT  TASK  TRIAL  DATA 

The  number  of  trials  received  and  the  number  of  trials  to  proficiency  in 
the  simulator  and  the  aircraft  were  obtained  for  all  0 group  students.  Data 
on  trials  were  collected  for  the  tasks  shown  in  table  11.  Table  11  shows 
trial  data  per  task  in  terms  of  the  number  of  students  achieving  P in  each 
task,  the  average  number  of  trials  to  proficiency,  the  number  of  students  not 
trained  to  P for  each  task,  and  the  average  number  of  trials  that  they  received. 
No  task  was  trained  to  proficiency  for  all  students,  nor  was  any  student 
trained  to  proficiency  in  all  tasks.  The  total  possible  cases  in  which 
students  could  be  trained  to  proficiency  was  624  (39  students  x 16  tasks). 

The  actual  cases  in  which  students  were  trained  to  proficiency  was  214,  or  34 
percent  of  the  cases.  This  is  contrasted  to  an  attainment  of  proficiency  in 
90  percent  of  the  cases  for  the  first  experimental  group,  E-1. 

The  average  number  of  trials  to  proficiency  for  a given  task  was  generally 
fewer  than  the  average  number  of  trials  received  on  the  same  task  by  students 
who  did  not  achieve  proficiency.  The  failure  of  students  to  attain  proficiency 
on  a given  task  can  be  attributed  to  student  ability,  ineffective  instruction, 
or  failure  of  the  instructor  to  assign  a P if  proficiency  was  attained. 
Unfortunately  for  the  0 group,  the  inexperienced  instructors  consistently 
awarded  fewer  Ps  than  did  the  experienced  instructors. 

AIRCRAFT  TRIALS  BY  TASK  AS  A FUNCTION  OF  PROFICIENCY  ATTAINMENT  IN  THE  2F87F. 

The  relationship  between  overall  simulator  performance  and  later  performance 
in  the  aircraft  was  discussed  in  section  II.  Table  12  presents  the  same 
relationsf'D  on  a per  task  basis.  On  most  tasks  if  proficiency  was  first 
attained  in  the  simulator,  the  number  of  trials  required  to  achieve  proficiency 
in  the  aircraft  were  fewer  than  if  these  tasks  were  not  trained  to  proficiency 
in  the  simulator.  The  difference  in  the  number  of  trials  is  not  significant, 
but  the  probability  of  a P in  the  aircraft  if  given  a P in  the  simulator  is 
high.  These  probabilities  are  shown  in  table  12.  The  group  trained  to  P in 


30 


TAEG  Report  No.  54 


I 

1 


i 

j 


TABLE  11.  SIMULATOR  TRIALS  RECEIVED  AND  SIMULATOR  TRIALS  TO  PROFICIENCY 


No.  of  Students 
Tasks  Attaining 

Proficiency 

Average  Trials 
to 

Proficiency 

No.  of  Students 
NOT  Attaining 
Proficiency 

Average 

Trials 

Received 

Brake  Fire 

3 

1 

36 

.8 

Abort  Four  Engines 

21 

1.5 

18 

2.1 

Abort  Three  Engines 

14 

2.9 

25 

7.5 

Engine  Failure  After 
Refusal 

17 

2.4 

22 

5.1 

Departure 

29 

2.3 

10 

3.5 

Holding 

14 

4.1 

25 

3.1 

TACAN/VOR 

18 

3.5 

21 

7.8 

NDB 

0 

- 

39 

.15 

LOC 

4 

1.8 

35 

.8 

GCA 

22 

2.2 

17 

3.4 

ILS 

7 

1.7 

32 

1.2 

Normal  Landings 
(Land  Flap) 

13 

10.2 

26 

13.5 

Approach  Flap  Landings 

12 

4.0 

27 

7.1 

Waveoff 

11 

1.8 

28 

4.5 

Three  Engine  Landings 

11 

4.1 

28 

4.0 

No  Flap  Landings 

18 

1.4 

21 

2.9 

the  simulator  equaled  or  bettered  the  group  not  trained  to  P in  the  simulator 
in  all  but  task  31.  Although  not  shown,  the  following  probabilities  were 
derived  from  table  12.  If  proficiency  is  first  attained  in  the  simulator, 
the  probability  of  attaining  proficiency  in  the  aircraft  is  .84.  Whereas  if 
proficiency  is  not  attained  in  the  simulator,  the  probability  of  attaining 
proficiency  in  tfi?  aircraft  is  .60.  One  is  cautioned,  however,  not  to  conclude 
that  the  probability  of  a P in  the  aircraft  associated  with  a P in  the  simulator 
is  due  solely  to  simulator  training.  The  possibility  exists  that  a P in  the 
aircraft  is  related  to  the  ability  of  the  student.  Determining  the  exact 
relationship  is  difficult.  This  is  discussed  in  section  IV  of  this  report. 

THE  LANDING  TASK  TRIAL  DATA.  Concurrent  with  the  introduction  and  acceptance 
of  visual  simulation  into  the  training  mission  of  simulators,  speculation 
exists  about  the  efficacy  of  visual  systems  for  training  the  landing  tasks. 
Despite  the  evidence  that  simulator  trained  students  required  fewer  landings 
in  the  aircraft  than  students  not  trained  in  the  simulator, 9 many  instructors 
expressed  doubt  concerning  the  effectiveness  of  Device  2F87F  for  training  the 
landing  task.  However,  based  on  evidence  to  date  Device  2F87F  with  its 
visual  system  is  more  effective  for  teaching  Normal  Landings  than  any  other 
task.  Table  13  compares  the  average  number  of  landing  trials  received  and 
the  average  number  of  trials  to  proficiency  for  the  E-2,  E-1 , and  0 groups. 

g 

Browning  et  al.,  op.  cit. 
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TABLE  13.  AIRCRAFT  LANDING  TRIALS  RECEIVED  AND  TRIALS  TO  PROFICIENCY 


Average  Trials 

Average  Trials 

Received 

To  Proficiency 

E-2  Group  (N  = 10) 

60 

50 

E-1  Group  (N  = 27) 

36 

17 

0 Group  (N  = 39) 

45 

28 

Both  the  E-1  and  0 qroups  received  fewer  trials  than  the  E-2  group. 

They  also  required  fewer  trials  to  attain  proficiency  than  the  E-2  group. 

Based  on  interviews  with  instructor  pilots,  the  differences  between  E-1  and  0 
group  landing  trials  are  most  likely  related  to  a more  rigid  grading  criterion 
imposed  by  instructors  of  the  0 group. 

Although  there  are  differences  between  the  E-1  and  0 group  in  the  number 
of  landings  required  to  achieve  proficiency,  the  evidence  indicates  that 
landing  practice  in  the  2F87F  provides  positive  transfer  of  training  to  the 
P-3  aircraft  regardless  of  variations  in  student  abilities,  differences  in 
grading  criteria,  or  instructor  experience. 

SUMMARY  OF  FINDINGS 

The  findings  discussed  in  this  section  are  summarized  below. 

1.  The  0 group  students  required  an  average  of  one  flight  hour  more 
than  the  E-1  group  to  complete  the  FAM/INST  phase  of  FRS  training. 

2.  Tasks  trained  to  proficiency  in  Device  2F87F  for  the  0 group  have  a 
higher  probability  of  being  judged  proficient  earlier  on  aircraft  flights 
than  tasks  not  trained  to  proficiency  in  the  simulator.  This  finding  is 
similar  to  the  results  obtained  earlier. 10 

3.  0 group  students  were  trained  to  proficiency  in  the  simulator  in 

only  34  percent  of  the  cases  as  compared  to  90  percent  for  the  first  experimental 
group  (E-l). 

4.  The  probability  of  attainment  of  proficiency  on  most  tasks  on  Fly  1 
was  lower  than  expected  since  these  tasks  had  been  previously  trained  in  the 
CFT,  CPT,  and  OFT  and  proficiency  demonstrated  in  either  the  CPT  or  OFT  (see 
table  12). 

Browning,  et  al.,  oo.  cit. 
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5.  The  probability  of  attaining  proficiency  in  a task  by  Fly  4 is  .84 
if  the  student  had  been  trained  to  "P"  in  that  task  in  the  2F87F.  This  is 
compared  to  a probability  of  .50  if  the  student  had  not  been  trained  to  "P" 
in  that  task  in  the  2F87F. 

6.  The  performance  of  the  0 group  again  demonstrated  that  Device  2F87F 
provides  positive  transfer  of  training  to  the  P-3  aircraft  for  every  task  in 
the  FAM/INST  phase  of  FRS  training. 

7.  The  2F87F  is  more  effective  for  training  landings  than  for  any 
other  task  in  the  syllabus. 
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SECTION  IV 

ADDITIONAL  CORRELATES  OF  PERFORMANCE  RELEVANT  TO  FRS  PILOT  PRODUCTION 

Previous  sections  of  this  report  examined  the  effectiveness  of  Device  2F87F 
for  maneuver/task  training  and  demonstrated  the  simulator's  capability  as  a 
direct  substitute  for  aircraft  training.  In  this  latter  determination,  training 
effectiveness  ratios  were  computed  to  provide  quantitative  indicants  of  tradeoff 
possibilities. 

During  this  study,  variables  beyond  those  formally  considered  in  the  evaluation 
were  identified  that  could  influence  training  decisions  in  the  production  of  P-3 
pilots.  This  section  describes  these  variables  and  their  effects  on  performance 
in  the  FRS  and  in  subsequent  assignments. 

Three  classes  of  relationships  are  examined  for  student  groupings: 

performance  (flight  grades  and  flight  hours)  in  UPT  and  subsequent 
performance  in  FRS, 

. performance  in  the  2F87F  OFT  and  its  effect  on  subsequent  performance 
in  the  air, 

FRS  performance  as  a predictor  of  performance  in  operational  assignments. 

Each  of  the  relationships  contribute  to  the  effectiveness  of  the  VP-30  FAM/INST 
phase  of  training. 

PERFORMANCE  IN  UPT  AND  SUBSEQUENT  PERFORMANCE  IN  FRS 

A VP-30  message  to  the  Commander,  Patrol  Wings  Atlantic,^ ^ provided  the 
impetus  to  investigate  the  relationships  between  UPT  flight  hours  and  UPT 
fight  averages  to  FRS  performance.  An  excerpt  from  the  message  is  provided 
below. 

"2.  WITH  THE  DELETION  OF  DEDICATED  SYLLABUS  INSTRUMENT  TRAINING  FLIGHTS 
AND  MAR  76  INTRODUCTION  OF  THE  2F87  OFT,  FRS  STUDENT  HOURS  HAVE  BEEN 
REDUCED  OVER  50  PER  CENT  SINCE  1973.  THE  ORIGINAL  PLANNING  FOR  THESE 
REDUCTIONS  WAS  BASED  ON  THE  ESTIMATED  CAPABILITIES  OF  THE  2F87  AND  AN 
UNDERGRADUATE  PILOT  TRAINING  (UPT)  STUDENT  INPUT  OF  260  HOURS.  WHEN  THE 
ACTUAL  FRS  REDUCTIONS  WERE  MADE,  CHANGES  IN  UPT  HAD  REDUCED  INPUT  HOURS 
TO  THE  CURRENT  AVERAGE  OF  205.  ALTHOUGH  IT  APPEARS  THAT  THE  CURRENT 
SYLLABUS  IS  MAINTAINING  FLEET  STANDARD,  THE  OPTIMUM  SIMULATOR  FLIGHT  MIX 
MAY  REQUIRE  ASSESSf*ENT." 

The  concern  of  the  FRS  is  understandable  if  UPT  students  with  fewer 
flight  hours  perform  poorly  compared  to  students  logging  a greater  number  of 
flight  hours.  However,  analysis  of  the  data  involving  59  students  indicated 
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that  concern  over  reduced  flight  time  upon  graduation  from  DPT  is  not  justified. 
Comparisons  and  correlation  of  (JPT  flight  hours,  UPT  flight  averages,  and 
VP-30  flight  averages  are  presented  in  table  14.  The  overall  VP-30  flight 
average  rather  than  the  Fly  4 check  grade  has  been  used  as  it  provides  a more 
stable  measure  of  performance  than  a one-time  measure  such  as  the  Fly  4 
check  flight. 


TABLE  14.  COMPARISONS  AND  CORRELATION  OF  UPT  PERFORMANCE,  UPT  FLIGHT  HOURS, 
AND  VP-30  PERFORMANCE 


UPT  Flight  Average 

Average  UPT  Flight  Hours 

VP-30  Average  Flight  Grade 

> 59 

197 

3.05 

50-59 

203 

3.04  (t=.78,  not  sig. 

at  .05  level ) 

< 50 

218 

2.92  (t=4.03,  sig.  at 

.01  level) 

Significance  Level 

UPT  Flight  Average 

vs.  UPT  Flight  Hours  -.59 

.05 

UPT  Flight  Average 

vs.  VP-30  Flight  Average  .50 

.01 

UPT  Flight  Hours  vs 

. VP-30  Flight  Average  -.29 

.05 

The  data  show  an  inverse  relationship  between  UPT  flight  grades  and  UPT 
flight  hours--the  greater  the  number  of  UPT  flight  hours  the  lower  the  flight 
grade  in  UPT.  The  same  relationship  exists  for  UPT  flight  hours  and  FRS  perfor- 
mance at  VP-30.  The  correlation  between  UPT  flight  average  and  FRS  flight 
average  is  significant  at  the  .01  level. 

A review  of  VP-30  flight  averages  beginning  in  1972  and  continuing  at  inter- 
vals to  the  present,  indicates  that  the  average  grade  has  been  about  3.03  with 
no  drop  coincident  with  the  decrease  in  programmed  UPT  flight  hours. 

During  the  TAEG  evaluations  it  was  noted  that  some  classes  required  more 
flight  hours  than  other  classes  even  though  UPT  class  averages  were  the  same. 

The  probable  explanation  is  that  in  a "lock  step"  curriculum  all  students  are 
scheduled  to  receive  at  least  four  flights  without  regard  to  performance.  Those 
students  who  have  problems  may  get  reflys  for  various  flights;  those  who  do  not 
pass  the  Fly  4 check  are  given  additional  checks  until  they  pass  or  are  set 
back  to  a later  class.  Generally,  these  are  students  with  UPT  flight  scores  of 
less  than  50.  The  students  with  high  UPT  flight  scores  that  perform  well  at  VP-3Q 
still  receive  a minimum  of  four  flights.  Thus,  for  a class  with  a large  variance 
in  UPT  flight  scores,  the  UPT  class  average  may  remain  near  the  historical  mean, 
but  the  FRS  flight  hours  will  vary  upward  due  to  performance  of  below  average 
students. 
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CORRELATION  BETWEEN  SIMULATOR  AND  FLIGHT  PERFORMANCE 

The  importance  of  attaining  proficiency  in  the  flight  simulator  on  each 
task  prior  to  training  in  the  aircraft  suggested  the  need  to  examine  correlations 
between  performance  in  the  simulator  and  later  performance  in  the  aircraft. 

The  results  of  these  analyses  are  presented  in  table  15. 


TABLE  15.  CORRELATION  OF  SIMULATOR  PERFORMANCE  AND  FLIGHT  PERFORMANCE 


Significance  Level 

VP-30  Simulator  Average  vs. 

Flight  Average 

.46 

.05 

Performance  in  the  Simulator 

on  Instrument 

Tasks  vs.  Performance  in  the 

P-3* 

.65 

.05 

* Instrument  tasks  include  holding,  precision  and  non-precision  approaches 
and  instrument  procedures. 


The  data  indicate  that  the  correlation  between  simulator  performance  and  later 
performance  in  the  aircraft  is  significantly  correlated  as  is  the  performance 
on  specific  instrument  tasks  to  later  performance  in  the  aircraft.  These 
findings,  while  not  cross  validated,  support  a conclusion  that  student  performance 
in  the  aircraft  can  be  predicted  with  some  certainty  based  on  his  performance 
in  the  simulator.  It  is  not  an  effective  training  strategy  to  take  a student 
to  the  aircraft  until  he  has  attained  proficiency  in  most  or  all  tasks  in  the 
simulator.  These  findings  augur  well  for  the  development  of  prescriptive 
training  strategies.  A course  of  instruction  can  be  tailored  to  the  student 
having  trouble  in  the  simulator  that  will  enhance  his  ability  to  benefit  from 
training  in  the  aircraft. 

FRS  PERFORMANCE  AS  A PREDICTOR  OF  PERFORMANCE  IN  OPERATIONAL  ASSIGNMENTS 

To  obtain  feedback  on  the  efficacy  of  FRS  training,  judgements  were  sought 
from  fleet  squadrons  on  the  operational  performance  of  assigned  VP-30  trained 
pilots.  Questionnaires  were  submitted  to  17  operational  squadrons  requesting 
information  on  the  performance  of  students  who  had  participated  in  the  earlier 
TAEG  studies  as  either  control  or  experimental  subjects.  Figure  4 contains 
sample  items  from  the  42  item  questionnaire. 

The  responses  of  36  respondents  to  the  questionnaire  were  analyzed  and 
the  results  are  presented  in  table  16. 
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Figure  4.  Sample  Questions  from  Followup  Questionnaire 
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TABLE  16.  SQUADRON  JUDGMENTS  OF  VP-30  FRS  CURRICULUM 
BASED  ON  STUDENT  PERFORMANCE 


GROUPS 


Ratinq  Scale 

Number  of 
Judgments 


CONTROL 

12  3 4 5 

11  142  637  31  1 


EXPERIMENTAL 
12  3 4 

7 133  507  15 


A 5-point  ratinq  scale  was  used: 

1 = Task  requires  much  more  emphasis 

2 = Training  less  than  adequate  for  task 

3 = Training  adequate  for  task 

4 = Training  more  than  adequate  for  task 

5 = Greatly  reduce  or  eliminate  task 


The  modal  response  for  the  control  and  experimental  groups  was  category  3 
(Training  Adequate)  of  the  ratinq  scale.  This  indicates  general  satisfaction 
with  the  VP-30  training  program.  More  importantly,  however,  was  the  finding  of 
no  difference  in  the  control  students  who  received  15.1  flight  hours  and  experi- 
mental students  who  received  8.6  flight  hours  (X^/3df  = 5.73).  In  addition  to 
the  overall  performance,  the  groups  were  also  compared  on  instrument  tasks  and 
landings.  The  comparisons  produced  no  significant  differences  (X^/2df  = 4.22 
for  instrument  tasks  and  = .86  for  landings).  There  were  too  few  responses 

for  other  tasks  to  make  valid  comparisons. 

PERFORMANCE  PREDICTORS 


Applicants  for  Navy  ang  Marine  Corps  pilot  training  are  given  various  selection 
tests;  among  these  are  the  Aviation  Qualification  Test  (AQT)  and  the  Flight  Aptitude 
Rating  (FAR).  Scores  on  these  tests  along  with  various  physical  and 
educational  criteria  are  used  in  the  selection  of  potential  candidates  for  pilot 
training.  Since  data  were  available  the  opportunity  presented  itself  for  a 
"quick  look"  at  the  relationships  between  the  AQT,  FAR,  and  performance  in  UPT 
and  FRS.  Correlations  were  computed  on  these  variables  with  a sample  of  65 
students.  The  results  are  presented  in  table  17. 

No  significant  correlations  were  found  for  any  of  the  combinations  examined. 
Based  on  this  sample,  the  AQT  and  FAR  scores  were  not  useful  predictors  of  FRS 
performance.  This  finding  is  consistent  with  findings  reported  by  North  and 
Griffin. 


R.  A.  North  and  G.  R.  Griffin.  Aviator  Selection  1919  - 1977.  Special 
Report  77-2.  1977.  (Naval  Aerospace  Medical  Research  Lal)oratory,  Pensacola, 

Florida)  p.  28. 
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TABLE  17.  CORRELATION  OF  AQT  AND  FAR 

TO  UPT  AND  FRS 

PERFORMANCE 

•"xy 

Multiple  r 

FAR  vs.  AQT 

.22 

FAR  vs.  UPT  Flight  Average 
(Basic  and  Advanced  Flight  Scores) 

.21 

AQT  vs.  UPT  Flight  Average 

.15 

FAR  vs.  FRS  Flight  Average 

.009 

AQT  vs.  FRS  Flight  Average 

.11 

FAR  and  AQT  vs.  UPT  Flight  Average 

.23 

FAR  and  AQT  vs.  FRS  Flight  Average 

.11 

SUMMMARY  OF  FINDINGS 

The  findings  discussed  in  this  section  are  summarized  below. 

1.  An  inverse  relationship  was  found  between  DPT  flight  averages  and 
UPT  flying  hours  and  between  FRS  flight  averages  and  UPT  flying  hours  (the 
greater  the  number  of  UPT  flight  hours,  the  lower  the  UPT  and  FRS  flight 
averages) . 

2.  Students  completing  UPT  with  a combination  basic  and  advanced 
flight  score  of  less  than  50  will  generally  be  expected  to  require  more  flight 
hours  to  complete  the  FAM/INST  phase  of  FRS  than  those  students  with  UPT  scores 
of  greater  than  50. 

3.  Based  on  the  UBAA  grades  assigned,  the  performance  of  first-tour 
students  in  Device  2F87F,  particularly  for  instrument  tasks,  is  predictive  of 
subsequent  performance  in  the  P-3  aircraft  (see  table  15). 

4.  Questionnaires  distributed  to  17  Fleet  squadrons  indicated  no 
significant  difference  between  experimental  and  control  students  after  leaving 
VP-30  in  (1)  overall  performance,  (2)  instrument  proficiency,  and  (3)  landing 
proficiency. 

5.  Based  on  the  limited  sample  examined  (N=65),  AQT  and  FAR  scores  were 
not  significantly  correlated  (at  the  .05  level)  with  performance  at  VP-30  and  are 
not  considered  as  predictors  of  performance  at  the  FRS  level. 


\ 
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SECTION  V 

CONCLUSIONS  AND  RECOMMENDATIONS 

This  section  presents  sets  of  general  and  specific  conclusions  from 
the  study.  For  each  specific  conclusion,  a course  of  action  is  recommended 
appropriate  to  the  finding. 

GENERAL  CONCLUSIONS 

1.  The  TAEG  P-3  studies  have  demonstrated  the  feasibility  of  onsite 
assessment  of  the  contributions  of  new  synthetic  devices  in  producing  aviators 
for  the  fleet.  In  this  case,  the  newly  installed  Device  2F87F  is  being  effic- 
iently integrated  into  the  ongoing  VP-30  training  system  without  interrupting 
or  delaying  the  pilot  production  commitments.  It  is  recommended  that  each  new 
major  device  undergo  formal  assessment  concurrent  with  its  introduction  to 
insure  effective  integration  into  an  ongoing  training  program. 

2.  To  maintain  the  effective  integration  of  a new  device  into  an  ongoing 
training  program,  certain  controls  are  required.  Among  these  are: 

a.  effective  employment  of  training  assets  that  matches  media 
capabilities  to  training  task  requirements;  i.e.,  CFT  and  CPT  for  part-task 
training  and  the  apportionment  of  OFT  and  aircraft  for  complex  whole  task 
training, 

b.  standardization  of  instructional  practices  and  grading  criteria, 

c.  instructor  training  in  the  capabilities  of  synthetic  trainers 
and  effective  integration  of  training  devices  into  the  training  continuum, 

d.  heightened  awareness  of  precise  management  control  requirements 
and  the  special  preparations  needed  for  efficiency  in  training. 

3.  In  support  of  the  effort  of  integrating  Device  2F87F  into  the  P-3 
curriculum,  additional  studies  are  needed  to  maximize  the  effectiveness  of 
training.  Foremost  among  these  are: 

a.  develop  performance  standards  and  an  automated  performance 
measurement  system, 

b.  determine  precisely  the  optimum  mix  of  simulator  and  aircraft 
training  to  achieve  FRS  FAM/INST  qualification.  To  achieve  this,  emphasis 
should  be  placed  on  determining  the  training  trials  required  as  a function  of 
student  past  performance  in  UPT. 

The  present  study  has  shown  that  device  effectiveness  is  in  part  depend- 
ent on  the  abilities  of  the  student  being  trained.  An  effective  training 
strategy  should  match  training  trials  or  periods  of  training  with  student 
abilities.  For  example,  six  simulator  periods  may  be  required  to  achieve  pro- 
ficiency for  some  students,  but  six  periods  may  be  inadequate  for  other  students, 
particularly  those  coming  from  UPT  with  lower  flight  scores. 
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SPECIFIC  CONCLUSIONS  AND  RECOMMENDATIONS 

CONCLUSION  RECOMMENDATION 


Due  to  the  material  condition  and 
utilization  practices  employed  with 
Device  2F69D,  it  was  not  signifi- 
cantly contributing  to  the  training 
of  dynamic  flight  tasks  immediately 
prior  to  and  after  acceptance  of  the 
first  Device  2F87F.  This  device  has 
become  a procedures  trainer. 

A review  of  the  Training  Device 
Systems  Summary  Report.  Level  1 
(Report  No.  DMMQIOLOI)'^  for  Device 
2F87F  Serial  Nos.  1 and  4 indicated 
an  unusually  high  number  of  fail- 
ures during  the  period  covered  by 
the  report. 


First-tour  students  with  average  UPT 
basic  and  advanced  flight  scores  of 
55  can  complete  the  Familiarization/ 
Instrument  phase  of  FRS  in  an  average 
of  15  flight  hours  in  the  P-3  without 
any  training  in  an  OFT. 

Comparison  of  the  performance  of  the 
experimental  group  trained  in  Device 
2F87F  and  the  P-3  with  performance 
of  the  experimental  group  trained 
in  the  P-3  without  2F87F  training 
has  demonstrated  the  Device  2F87F 
can  substitute  for  6.5  hours  of 
P-3  training.  The  performance  of 
the  operational  (0)  group  and 
students  subsequently  trained 
has  shown  that  the  average  training 
time  is  increasing.  It  is  concluded 
that  this  gradual  increase  in  flight 
hours  will  continue  unless  positive 
and  aggressive  action  is  taken. 


The  device  should  be  restored  to  its 
designed  capability.  Past  evidence 
has  shown  that  the  device  provides 
excellent  training.  Effective  utili- 
zation of  the  2F69D  could  relieve  some 
of  the  pressure  on  2F87F  utilization. 


Both  2F87Fs  should  be  checked  by  des- 
ignated qualified  pilots  on  a regular 
basis  to  insure  maintenance  of  simula- 
tion and  consistency  of  performance 
between  the  two  devices.  Further, 
the  quality  of  maintenance  should  be 
improved  to  assure  availability  of 
all  systems  for  every  training  period. 
Vigorous  action  is  essential  to  pre- 
vent the  2F87F  from  being  utilized 
essentially  as  a procedures  trainer. 

Fifteen  flight  hours  should  be  used 
as  a basis  for  scheduling  when  the 
simulator  is  unavailable  for  training 
due  to  maintenance  or  modification. 


Provide  more  comprehensive  training 
for  all  new  instructors  in  effective 
utilization  of  Device  2F87F.  Schedule 
each  new  instructor  to  observe  an 
experienced  instructor  for  at  least 
one  class  before  being  allowed  to 
train  students  in  the  simulator  or 
aircraft.  Establish  grading  criteria 
based  on  defined  standards  of  performance 
and  require  adherence  to  these  criteria. 


Published  by  Code  N-434,  Naval  Training  Equipment  Center,  Orlando,  FL. 
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CONCLUSION 


RECOMMENDATION 


The  performance  data  of  this  study 
strengthens  and  supports  earlier 
findings  that  landing  practice 
in  Device  2F87F  transfers  positively 
to  the  aircraft. 

Students  trained  to  proficiency  on 
tasks  in  the  simulator  have  a higher 
probability  of  performing  the  task 
to  proficiency  on  the  first  and 
succeeding  flights  in  the  aircraft. 


Performance  in  Device  2F87F  is  signif- 
icantly correlated  with  later  per- 
formance in  the  P-3. 


Students  who  have  not  demonstrated 
proficiency  on  all  tasks  are  being 
recommended  for  Fly  4 checks.  Some 
then  require  a second  or  third  refly 
of  the  Fly  4 check. 

Concern  over  students  coming  to  the 
FRS  with  fewer  flight  hours  than  the 
historical  average  is  unfounded  based 
solely  on  UPT  flight  hours.  The 
study  results  indicate  an  inverse 
relationship  between  UPT  pilot  hours 
and  UPT  flight  grades. 

UPT  basic  and  advanced  flight  scores 
are  valid  predictors  of  performance 
at  VP-30.  Incoming  students  with  a 
UPT  flight  score  of  less  than  50  can, 
on  the  average,  be  expected  to  require 
more  flight  time  at  VP-30  and  to 
finish  with  scores  lower  than  the 
3.03  average  established  at  VP-30 
over  a number  of  years. 


Disseminate  this  information  to  all 
instructors  to  ensure  development  of 
more  positive  attitudes  toward  the 
simulator. 


Students  should  not  be  scheduled  in 
the  aircraft  until  proficiency  has 
been  attained  in  the  task  planned  for 
training  in  the  aircraft.  Seek  to 
return  to  block  training  in  the  simu- 
lator with  a requirement  that  all  tasks 
be  trained  to  proficiency  in  the 
simulator  prior  to  any  aircraft 
training. 

Use  a prescriptive  approach  to  train- 
ing. For  the  student  experiencing 
trouble  in  the  simulator,  emphasize 
training  that  will  correct  deficiencies. 
Extend  simulator  training  if  necessary. 

For  students  not  proficient  at  the 
end  of  Fly  3,  schedule  a Fly  4 and 
Fly  5,  if  necessary,  instead  of  a 
check  flight  for  which  they  are  not 
prepared. 

Analyze  each  incoming  student's  UPT 
performance  record  and  prescribe  a 
syllabus  based  on  expected  accom- 
plishment. 


Develop  a FAM/INST  curriculum  that 
will  provide  training  that  can  accom 
modate  differing  learning  rates 
of  students. 
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ADF 

AIRCND/PRSR 

AQT 

BASIC  ARWK 
CFT 

COPILOT  RESP 

CPT 

EFAR 

ELECT  SYS  OP 
EMERG  DESCENT 
EMERG  SHTWN 
ENG  RSTRT 
FAM/INST 
FAR 

FIRE  UNK  ORIG 
FRS 

FUEL  SYS  OP 
GCA 

HYD  SYS  OP 
ILS 

INST  PROCEDURES 
KNWLG  PROCEDURES 
LANDING  PTRN  AIRWK 
LOC 

LOITER  SHTWN 
MISSED  APP 
NAV  INST  FAIL 
NDB 

NON  PREC  APP 

NTS 

OFT 

P 

P-3 

PROP/ENG  MALE 

RES  ELECT  PWR 

TACAN 

TAEG 

TER 

UPT 

VOR 

VP-30 


GLOSSARY 

Automatic  Direction  Finding 
Airconditioner/Pressurization 
Aviation  Qualification  Test 
Basic  Airwork 

Cockpit  Familiarization  Trainer 
Copilot  Responsibilities 
Cockpit  Procedures  Trainer 
Engine  Failure  After  Refusal 
Electrical  System  Operation 
Emergency  Descent 
Emergency  Shutdown 
Engine  Restart 
Familiarization/Instrument 
Flight  Aptitude  Rating 
Fire  of  Unknown  Origin 
Fleet  Readiness  Squadron 
Fuel  System  Operation 
Ground  Controlled  Approach 
Hydraulic  System  Operation 
Instrument  Landing  System 
Instrument  Procedures 
Knowledge  of  Procedures 
Landing  Pattern  Airwork 
Localizer  Approach 
Loiter  Shutdown 

Missed  Approach  ^ 

Navigation  Instrument  Failure  « 

Non-di recti onal  Beacon 
Non-precision  Approach 
Negative  Torque  Sensing 
Operational  Flight  Trainer 
Proficiency 

Lockheed  Orion  Aircraft 

Propeller/Engine  Malfunction 

Restoring  Electrical  Power 

Tactical  Air  Navigation 

Training  Analysis  and  Evaluation  Group 

Transfer  Effectiveness  Ratio 

Undergraduate  Pilot  Training 

Very  High  Frequency  Omnidirectional  Range 

Patrol  Squadron  THIRTY 
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